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ABSTRACT 

Allegations by a resigning employee of intimidation, 
cover-up, unethical behavior, legal violations, waste, and 
mismanagement were investigated at a major university, relying on 
confidential interviews ana a review of relevant documentation. The 
case study of this evaluation illustrates the advantages of a mix of 
qualitative and quantitative approaches in evaluation and audit. 
Confidential interviews with 16 health and safety professional staff 
members were charted to graphically illustrate staff concerns and 
were categorized for relevant issues. Ihe external view of the 
evaluator was juxtaposed against the internal views ot staf i . Both 
internal and external evaluator s found evidence of some waste and 
mismanagement. The external evaluator thought that there had been one 
instance in which the university acted unethically, but found no 
evidence of a cover-up; staff members recognized no examples of 
unethical action, but some believed there might have been a cover-up. 
These apparent contradictions represented weaknesses in communication 
among all parties, rather than the lack of reality of the insiders 1 
viewpoints. Combining quantitative and qualitative approaches 
enhanced the usefulness and accuracy of the evaluation. Three 
illustrative figures are provided. (SLD) 
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From Qualitative to Quantitative and Back Again: 
Philosophical and Methodological Transitions 

I am a firm believer in the value of a good qualitative and quantitative mix in evaluation 
and audit. Combining approaches naturally enhances the process and outcome of a study. 
Moreover, it is douDtful that any study can be conducted well without using both qualitative 
and quantitative methods. Attempting to conduct a purely qualitative or quantitative evaluation 
is artificial and needlessly limits the use of potentially rich data sources. In addition, failure to 
maximize the full range of tools in the trade may weaken an argument and diminish an evalua- 
tor's credibility, resulting in a failure to convince a client of the utility of an evaluation find- 
ing, and seriously eroding the study's impact. 

Evaluators typically have a predisposition toward the use of qualitative or quantitative 
methods, based on their education and experience. However, the evaluator's predisposition and 
the dominance of an approach in a given study does not suggest exclusivity, lack of rigor, or 
the complete absence of any other approach. The nature of A " ? problem and the audience 
determine the best way to present evaluation and audit findings — not the intrinsic value of 
any approach per se. 

This case is an example of a qualitative/quantitative mix from a highly publicized evalua- 
tion. The evaluation required some methodological dexterity: methodological shifts from quali- 
tative to quantitative and back again were necessary to complete a meaningful and useful study. 
The study involved collecting confidential interviews, converting them into quantitative data, 
reporting the data in a quantitative (and qualitative) mode, and then assessing the quality of the 
quantitatively presented information. The philosophical positions underlying these methodologi- 
cal transitions from phenomenology to positivism and back again also come under discussion. 
The Case Study 

I was asked to evaluate allegations of intimidation, cover-up, unethical behavior, legal viola- 
tions, and waste and mismanagement at a major university. These allegations were made by a 
senior health and safety officer in his letter of resignation. 

This study relied primarily on confidential interviews and a review of relevant documenta- 
tion. I reviewed thousands of sheets of information, ranging from electronic mail to newspaper 
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stories. I cross-checked information using a wide variety of sources. As I conducted interviews 
and reviewed documents, I discussed my tentative conclusions wi<h those people directly affect- 
ed. I subsequently wrote internal memoranda about specific subjects and again shared those 
memoranda with the employees they concerned, encouraging them to provide corrections or 
supplementary information. 

The heart of this discussion involves the confidential interviews I conducted with staff 
members at the university. I interviewed all sixteen health and safety professional staff mem- 
bers, and their responses provided a useful barometer with which to gauge how broadly these 
concerns were shared. Due to the sensitive nature of this evaluation, special precautions were 
taken to safeguard all information and to protect the confidentiality of concerned staff. Sensi- 
tive interviews were carefully coded to preserve anonymity. (See Fetterman 198£ for details 
about the study.) 
The Insider's View 

These confidential interviews provided an insider's view or emic perspective about these 
allegations. The ex-employee felt so frustrated by what he perceived as the university's lack 
responsiveness to problems he identified that he seemed to feel that the failure to accept his 
recommendations and act on them in the time frame he h^d recommended constituted "unethical 
behavior." I asked each of the professional staff members in the department if they believed 
the university had engaged in any unethical behavior. Despite the fact that 1 had invited them 
to define "unethical behavior" as broadly as they wished, none of them knew of any case or 
believed that there were any cases where the university had n cted unethically. (See Figure 1.) 

The former employee had been instrumental in creating a computer data base in which 
department staff members can enter any health and safety problem they identify. In principle, 
after completing an inspection, a health and safety professional enters the data, including a 
description of the problem, its location, and its severity, as well as the university department 
believed to be responsible for correcting the condition. The entries are printed out and sent to 
the responsible department, and there is a field in each record for entering the date when the 
department corrects the problem. 
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A number of professionals, including the former employee, diligently recorded problems, 
but others did so only intermittently or not at all. There has been, moreover, very little follow- 
up on the identified hazards: The health and safety staff have not checked with the depart- 
ments to find out what, if anything, has been done (19% of the professional staff were aware of 
code violations, primarily in this data base, that were being ignored; see Figure 1) -- nor have 
the departments, typically, reported back to the health and safety department. Of almost 5,300 
entries in the data base, only 173 record completion dates. Others have certainly been corrected 
as well, but the numbers are unknown due to lack of follow-up. In general, there is little 
feedback to the staff concerning responses to their recommendations, or time lines for correct- 
ing problems they identify. Thus, having heard nothing of steps taken or planned, some em- 
ployees bJieved there might be a cover-up (See Figure 1). 

The former employee cited numerous examples of ways in which he believed he was "intim- 
idated." However only one of them constituted intimidation. After being informed that the 
former health and safety professional had contacted the press about problems with the universi- 
ty's incinerator, the supervisor said the former employee should be told that he would be fired 
if he wen. to the press again. I found this case to be unique: No other employee reported 
being threatened with discharge if he or she were to disclose problems. (See Figure 1.) 

I did find mismanagement and waste on both the university and health and safety depart- 
ment levels. This finding results from the entire investigation, including not only the a M na- 
tions of the former employee, but other matters that emerged in the course of the study. As 
Figure 1 shows, 75% of the health and safety department staff believed there was university 
level mismanagement and waste, and 88% stated there was departmental mismanagement and 
waste. 

The Quantitative Conversion 

These qualitative confidential interviews were converted into figures and a bar chart to 
graphically illustrate department staff member perceptions -- a useful baseline in this sensitive 
and broad investigation. The information provided policymakers and the public with a handle 
on the widespread character (the representativeness), of these concerns among a knowledgeable 



university population. 

The long verbatim interviews were reduced to summaries and key statements that indicated 
the speakers* views about a specific topic. (See Figure 2.) The number of individuals who 
agreed with a specific allegation were added together, and that figure was divided by the total 
number of the staff to calculate the percentages that appear in the bar chart. (See Figure 3.) 

Most interviews were easy to categorize. They were straightforward, honest comments 
about the relevant issues. However, interviewees are not always clear communicators. In some 
instances, an external, social scientific, judgment was needed to properly categorize these (phe- 
nomenologically derived) insider perceptions about the allegation topics. For example, one staff 
member replied n£ to my question about whether he was aware of any legal or code violations. 
However, he then spent the next half hour detailing all the specific exceptions to this initial 
statement. (See Figure 2.) In essence, he said nfi, a pro forma response to an independent 
interviewer, and then as trust and rapport developed, he opened up about all the legal violations 
he was aware of in the university. Thus I classified his response a ves . instead of a nQ. (I also 
triangulated information to verify his statements about the violations.) Thus, the data reported 
in the bar chart more accurately reported the insider's perspective with an external qualifier 
than it would have at face value. 
Ihfi External (Social Scientifi v View 

An external (social scientific) assessment was needed to place the insiders* views in context, 
even after the insiders' views were qualified and categorized by external review. In some cases, 
there was a conflict between external positivistic assessments and primarily phenomenological 
insider information. However, this conflict was typically a result of differing levels of knowl- 
edge about a given issue. For example, none of the health and safety staff members knew 
about any case or believed that there were any cases in which the university acted unethically. 
However, I felt that in one case the university had acted unethically, although it was a close 
call. 

The university used two incinerators to dispose of some of its infectious waste materials. 
There ,iad been a histor/ of intermittent compi "nts about foul odors emanating from one of the 
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Legal Violations 

Interview A, C, D, F, G, H, I, J, K, L, N, O, P: 
No legal violations that they were aware of. 

★ 

Interview B, E, M: 

★ 

B: we have many violations. Every steam 
tunnel... 

★ 

E: said there wasn't a legal violation, but then 
contradicted self with examples of compliance 
violations, e.g. chemical waste storage... 

★ 

M: does not believe we are in compliance 
with... 
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inhersity's incinerators. Employee complaints about respiratory distress and elevated levels of 
illness focused on the second incinerator. Although the tests performed by the health and safety 
department did not isolate any specific agents or organisms that could be linked o the reported 
symptoms, management knew of the reentrainment (fumes coming back into the office area), 
and knew that it might have contributed to those symptoms. At the same time, the university 
was building a new incinerator to replace these incinerators. 

I believe that some time after the university incinerator problems had been identified as 
serious, the university should have stopped using these incinerators altogether (and used off- 
campus facilities until the new incinerator was operational) or, at very least, given written 
notice of the possible hazards to all employees working in areas that mi^ht have been affected 
(rather than relying on the haphazard communications that occurred). 

The cover-up issue represented another area of apparent disagreement ok misinterpretation. 
The health and safety database was by no means an attempt to conceal but, just the contrary, an 
attempt to communicate problems to those who should deal with them. The failure to follow up 
is an example of ineffective institutional man?gement: There was a lack of a systematic proce- 
dure for assuring that identified problems were corrected. 

Health and safety staff thought that there might be a cover-up, because there was no 
feedback loop to inform them about what had been done with their information. They did not 
know what was and was not being done in response to their findings. This was a serious 
communication problem. I recommended that health and safety staff members be informed 
about whether their recommendations are going to be adopted; if not, why not; and what priori- 
ty specific problems will receive. 

I did not disagree with the former employee or the health and safety staff members about 
the institutional intimidation issue. However, I thought it necessary to provide seme phenome- 
nologically based or insider's view of the context of that finding. 

The former employee viewed his job as one of problem identification and expected simple, 
rigid compliance with his recommendations. He viewed himself as an expert in his area who 
performed his job with precision and conformity to rules and regulations. Some of his col- 
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leagues viewed him as the consummate professional. According to other colleagues and clients, 
he had a tendency to "overreact," providing "the worst-case scenario." Few people doubted his 
technical precision, but even colleagues who admired his professional abilities questioned his 
manner of presenting his findings. 

The former employee identified many health and safety hazards, ranging torn serious prob- 
lems to minor ones. Unfortunately, he presented the problems without providing a sense of 
proportion. His presentation created the imp ^ion that all the problems he identified were of 
equal significance and priority. Clients who requested guidance in assigning priorities to the 
problems complained that little help was forthcoming. 

This failure to establish priorities was compounded by incidents in which some faculty 
clients felt that he used "scare tactics" to communicate his findings. For instance, in pointing 
out unsatisfactory laboratory practices, the former employee often told people that all members 
of the department faculty could go to jail. He had been criticized in the past for the manner of 
his communication with other university departments, which was perceived by the other de- 
partments as efforts to frighten them rather than to provide help and advice. 

Moreover, some members of management felt that he was simply saying the same things 
over and over again and that he had "no new news" to report. Management felt it had taken his 
advice into account, weighed the options along with other considerations, and made decisions 
accordingly. 

I also felt that an assessment of this situation required some discussion about the role of a 
health and safety officer. This role is filled with a variety of obligations that may be in tension 
with, or even conflict with, one another. A health and safety officer is charged with identify- 
ing, documenting, cataloging, and responding to problems. Management, however, is responsi- 
ble for establishing priorities and time lines to respond to identified hazards. Management's 
response is based on the severity of the problem, availability of resources, and systemwide plans 
and activities. A difference of opinion often exists between management and a health and 
safety officer about the urgency required to respond to a given hazard. Arguments abound 
between interested parties, each playing its part in a process in which resources are not adequate 
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to respond to all needs. 

I concluded that no organization can function effectively under siege -- in fear of its own 
employees' appealing to the press every time they feel problems are not being resolved in the 
way they have recommended and within their personal time lines. In this case, management, 
rightly or wrongly, believed that the former employee was attempting to circumvent the organi- 
zation's decision-making process. But threatening to fire him was nonetheless an act of intimi- 
dation. 

In summary, my (external) view was juxtaposed against the health and safety staff member 
(insider) views in the areas of unethical behavior and cover-up. None of the staff members 
thought the university had acted unethically. I thought one case raised ethical considerations. 
Some professional staff members (31%) believe that there might be a cover-up. I understood 
why they thought there might be a cover-up. However, I found no instances of a cover-up. In 
addition, my (external) assessment of the intimidation issue was counterpoised with my (insider) 
view or phenomenologically derived perception of management decision making. This phe- 
nomenologically based information was used to meaningfully communicate the context of this 
aberrant case of intimidation. 

These contradictions between the insiders' views and my own external view do not imply 
that the insiders' views are any less real tc the insiders or that they do not shape their behavior. 
Rather, the contradictions point io u weakness in the communication loop between all parties 
concerned. 

Conclusion 

The journey from qualitative to quantitative and back again may appear circuitous at times. 
However, it iz a logical progression toward an accurate portrayal of events. (See Fetterman 
1989.) Stopping prematurely at one stage in the process can be misleading -- providing a false 
picture of the entire event. 

The nature of the problem and the experience cf the evaluator will dicfate which approach 
is fundamentally best suited to the task However, combining both qualitative and quantitative 
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approaches at appropriate junctures will only enhance the usefulness and accuracy of an evalua- 
tion. 
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